In imperfect-information games, the optimal strategy in a subgame may depend on the strategy in other, unreached subgames.

In imperfect-information games, the optimal strategy in a subgame may depend on the strategy in other, unreached subgames. Thus a subgame cannot be solved in isolation and must instead consider the strategy for the entire game as a whole, unlike perfect-information games. Nevertheless, it is possible to first approximate a solution for the whole game and then improve it in individual subgames. This is referred to as subgame solving. We introduce subgame-solving techniques that outperform prior methods both in theory and practice. We also show how to adapt them, and past subgame-solving techniques, to respond to opponent actions that are outside the original action abstraction; this significantly outperforms the prior state-of-the-art approach, action translation. Finally, we show that subgame solving can be repeated as the game progresses down the game tree, leading to far lower exploitability. These techniques were a key component of Libratus, the first AI to defeat top humans in heads-up no-limit Texas hold'em poker.

Unlike perfect-information games, imperfect-information games cannot be decomposed into subgames that are solved independently. Thus more computationally intensive equilibrium-finding techniques are used, and abstraction---in which a smaller version of the game is generated and solved---is essential. Endgame solving is the process of computing a (presumably) better strategy for just an endgame than what can be computationally afforded for the full game. Endgame solving has many benefits, such as being able to 1) solve the endgame in a finer information abstraction than what is computationally feasible for the full game, and 2) incorporate into the endgame actions that an opponent took that were not included in the action abstraction used to solve the full game. We introduce an endgame solving technique that outperforms prior methods both in theory and practice. We also show how to adapt it, and past endgame-solving techniques, to respond to opponent actions that are outside the original action abstraction; this significantly outperforms the state-of-the-art approach, action translation. Finally, we show that endgame solving can be repeated as the game progresses down the tree, leading to significantly lower exploitability. All of the techniques are evaluated in terms of exploitability; to our knowledge, this is the first time that exploitability of endgame-solving techniques has been measured in large imperfect-information games.

When solving extensive-form games with large action spaces, typically significant abstraction is needed to make the problem manageable from a modeling or computational perspective. When this occurs, a procedure is needed to interpret actions of the opponent that fall outside of our abstraction (by mapping them to actions in our abstraction). This is called an action translation mapping. Prior action translation mappings have been based on heuristics without theoretical justification. We show that the prior mappings are highly exploitable and that most of them violate certain natural desiderata. We present a new mapping that satisfies these desiderata and has significantly lower exploitability than the prior mappings. Furthermore, we observe that the cost of this worst-case performance benefit (low exploitability) is not high in practice; our mapping performs competitively with the prior mappings against no-limit Texas Hold'em agents submitted to the 2012 Annual Computer Poker Competition. We also observe several paradoxes that can arise when performing action abstraction and translation; for example, we show that it is possible to improve performance by including suboptimal actions in our abstraction and excluding optimal actions.

When solving extensive-form games with large action spaces, typically significant abstraction is needed to make the problem manageable from a modeling or computational perspective. When this occurs, a procedure is needed to interpret actions of the opponent that fall outside of our abstraction (by mapping them to actions in our abstraction). This is called an action translation mapping. Prior action translation mappings have been based on heuristics without theoretical justification. We show that the prior mappings are highly exploitable and that most of them violate certain natural desiderata. We present a new mapping that satisfies these desiderata and has significantly lower exploitability than the prior mappings. Furthermore, we observe that the cost of this worst-case performance benefit (low exploitability) is not high in practice; our mapping performs competitively with the prior mappings against no-limit Texas Hold'em agents submitted to the 2012 Annual Computer Poker Competition. We also observe several paradoxes that can arise when performing action abstraction and translation; for example, we show that it is possible to improve performance by including suboptimal actions in our abstraction and excluding optimal actions.